The present invention generally relates to the use of a visual reference to prevent apparent motion sickness caused by conflicting sensed cues, and more specifically, to a method and apparatus that visually indicate a background reference corresponding to the vestibular senses of the inner ear.
A problem commonly experienced by people traveling in a vehicle, aircraft, or boat is motion sickness. Motion sickness occurs because of a mismatch between the sensations felt in the inner ear vestibular system and those experienced through other senses, such as a person""s visual perceptions. It should be noted that for purposes of this discussion and as used in the claims that follow, it will be understood that the vestibular system of the inner ear is an xe2x80x9cinertial motion sensor.xe2x80x9d Balance receptors in the inner ear respond to gravity (e.g., changes in orientation), velocity, and changes in velocity (accelerations) experienced while moving. When the sensations experienced by the inner ear fail to match visual cues, motion sickness often results.
For example, a passenger traveling along a winding road in an automobile experiences linear and angular accelerations each time the vehicle travels around a curve. The response of the vestibular sensing system to the acceleration caused by the motion of the vehicle will not match the visual perception unless the person is continually viewing the road so that the perception of the person""s inner ear corresponds to the visually perceived path of the vehicle around curves. It is for this reason that the driver of a vehicle normally does not experience motion sickness, even though motion sickness may result when the person is traveling as a passenger in the vehicle driven along the same road and in the same manner, by someone else. A driver continuously watches the road and visually perceives the motion of the vehicle so that the visual perceptions match the senses of the inner ear. Passengers in a vehicle, who are reading, or only viewing the interior of the vehicle, or carrying on other activities that cause them not to watch the road, will have a visual perception that does not match the senses of their inner ear. As a consequence, the passengers may experience symptoms associated with motion sickness, such as nausea, headache, and disorientation. Most people find that while traveling as a passenger, they can generally avoid motion sickness by watching the road as if they were driving the vehicle, so that their visual input more closely matches the sensations of their vestibular system. However, when confined within an interior cabin of a ship or in other situations in which the actual movement of the person experienced by the vestibular inner ear apparatus cannot readily be related to the visual perception of that movement, motion sickness is not so easily avoided.
The equivalent of motion sickness can also be experienced when a person is not moving, but the visual input experienced by the person appears to indicate that the person is moving. When a person is in an earth-fixed, and non-moving environment, the mismatch between an apparent self-motion that is evoked visually and the sensation by the inner ear indicating that no environment motion is actually being experienced can cause the symptoms of motion sickness. The conditions leading to this problem can occur, for example, when viewing an IMAX(trademark) motion picture presentation (i.e., a movie displayed on a wrap-around screen) in which motion is portrayed, or when participating in a flight or driving simulation running on a computer, or when viewing moving images in an artificial environment produced with a head-mounted display (HMD). Accordingly, as used throughout this disclosure and in the claims that follow, the term xe2x80x9cmotion sicknessxe2x80x9d is intended to encompass any of the symptoms commonly associated with being in a moving environment, as noted above, as well as corresponding symptoms experienced by persons who are not moving, but are exposed to a virtual environment in which motion is displayed, causing the perception of motion by the person.
The unpleasantness associated with motion sickness during travel can be sufficient to cause a person to refuse to travel under the conditions that produce motion sickness. Similarly, the motion sickness that can result when interacting in a simulator or viewing a virtual environment presentation may preclude a person from viewing a presentation on a surround screen or from participating in a simulator or virtual environment experience. It would be unfortunate, when such visually stimulating experiences are becoming more readily available and more refined due to advances in graphics and display technology, that concern about motion sickness might prevent someone from enjoying these experiences. Accordingly, a solution is needed that will enable a person to participate in activities where visual scene motion may evoke illusory self-motion without concern that the symptoms of motion sickness will be felt, or at least, that the severity of such symptoms can be controlled. Ideally, any solution that avoids these symptoms should also be applicable to preventing actual motion sickness caused when a person is in motion, as described above.
Recognizing that motion sickness is caused by a sensory mismatch, a solution purportedly addressing this problem is described in U.S. Pat. No. 5,966,680 (Butnaru). This patent discloses the use of a display on which xe2x80x9can artificial labyrinthxe2x80x9d is presented. The artificial labyrinth is displayed in an HMD or as a projection onto the retina and comprises an array of lines that are controlled to indicate movement of the user""s head in regard to a baseline position. The reference teaches that this indication should be updated at least every 150 ms. The artificial labyrinth includes visual cues that show changes in roll, pitch, yaw, and elevational position of the user""s head. To sense these changes in position, head-mounted gyroscopic, accelerometers, or magnetostrictive sensors are employed. The signals produced by the sensors are input to a microprocessor that controls the display of the visual cue lines directly in response to the sensor output signals. The patent also teaches that a global positioning satellite (GPS) receiver can be included to provide an orientation or location signal that is visually presented to a user. The cue lines indicative of the user""s orientation relative to the baseline are displayed on transparent lenses of an HMD, or projected onto corrective or plain lenses of glasses, or projected into space a few feet in front of the user as a holographic display, or projected onto the retina of the user.
Alternatively, the Butnaro patent teaches that a camera and a projector can both be mounted on a glasses frame worn by the user and used to produce a recorded image of a scene in front of the user. The recorded image is conveyed to a microprocessor and averaged with other images produced by the camera to produce a slowly changing display of the user""s environment. The slowly changing display is projected onto the lenses of the glasses, so that the user only perceives the averaged image as displayed by the projector on the inner surface of the lenses. The patent also indicates that a less desirable technique senses orientation with sensors that are not mounted to the user, but are instead responsive to the orientation of an environment (such as an automobile, aircraft, or boat) in which the user is traveling.
There is a substantial problem with the solution proposed by Butnaro to prevent motion sickness. Butnaro teaches that the signals produced by head-mounted accelerometers or other motion sensors are directly indicative of the observer""s true orientation and motion and thus, that these signals correspond to the sensations of the inner ear. Butnaro apparently fails to fully appreciate that some motion tracker sensors do NOT produce signals indicative of the true orientation and motion to which the devices are subjected. For example, accelerometers do not sense true constant linear or angular velocities, whereas an inertial tracker may do so. Butnaro apparently fails to fully appreciate that the response of the vestibular receptors to motion does NOT correspond to that of head-mounted accelerometers or other motion sensors. As will be clear from the discussion that follows, tracker/accelerometer signals must be manipulated appropriately so that simulated vestibular signals match real ones. Finally, Butnaro apparently fails to fully appreciate that the response of the vestibular receptors often does NOT reflect the true orientation and motion of a person""s head and body. Indeed, it is very possible that a mismatch between the visual cues provided by the Butnaro system and the perceived sensations of a user""s vestibular system might tend to cause motion sickness rather than prevent it. For example, a person may be learning a task in a simulation while in a moving environment such as a car, airplane, or centrifuge. Butnaro proposes to provide a visual reference that matches the person""s true orientation and motion. However, if the real angular motion exceeds the vestibular semicircular canal long time constant (as further discussed below), the vestibular signal will indicate that the person is stationary. Presentation of a visual reference indicating the true rotational motion of the person would be in conflict with the perception of that person""s vestibular receptors, and this conflict between the vestibular perception and Butnaro""s visual reference would be likely to exacerbate rather than alleviate motion sickness. The rotating environment case may be an extreme example, but nevertheless illustrates the fundamental issue with the disclosure of this prior art patent. More subtle cases illustrating problems with the reference are associated with time delays and phase shifts between tracker signals and vestibular perception. Time delays and phase shifts are known to evoke motion sickness.
Another example to illustrate discrepancies between vestibular perception and true motion is otolith ambiguity. This ambiguity can occur because the sensation of the otolith organs (in the inner ear) may reflect either linear acceleration or head tilt with respect to gravity. The ambiguity is apparent in the illusion of upward pitch experienced by pilots during forward linear acceleration in higher performance aircraft. The system disclosed by Butnaro would not take this ambiguity into account.
It would therefore be preferable to provide visual cues that truly match the perceptions of the vestibular organs, even though to do so may require that a signal indicative of the actual physical motion and acceleration experienced by a user be modified in accord with a vestibular model, to correspond to the true perceptions of the vestibular system. Also, it is possible that the model may need to be refined to achieve optimum results. For example, it may also be necessary to compensate for the affects of aging or other variability in the perceptions of users when providing a visual reference that corresponds to the sensations of the inner ear for a specific person. For this reason, any system and model attempting to match a visual perception with the perception of the vestibular organs should be capable of reprogramming and refinement to fine tune the match between the visual cues and the perceived motion by the person""s vestibular system. Butnaro""s disclosed invention does not have the capability of providing such compensation or modifying the visual cues to account for differences between individuals.
In addition, the invention disclosed in the above-referenced Butnaro patent is unlikely to be of significant benefit if the user is in an earth-fixed environment where visual scen motion causes the person to feel that he/she is moving, such as can occur when viewing a surround screen motion picture presentation, or when viewing and participating in a driving/flight simulator or virtual environment display. Also, it is probably impractical to require a user to continually wear an HMD or other display device while traveling for an extended period of time. In some cases, it will be preferable to provide an independent visual background (IVB) providing the required visual cues that are projected onto an interior surface of a cabin of a boat or other conveyance, or which is included in a surround presentation or in the background on a display so as to provide a visual reference corresponding to the perception of a person""s inner ear vestibular system. One advantage of such an IVB reference is that a plurality of people to which the display surface is visible will all benefit from viewing the reference.
Another problem with Butnaro and other prior art techniques that attempt to provide a visual reference to avoid motion sickness is that the visual reference can be too distracting. For example, in the technique disclosed in the Butnaro patent, it is clear that the cue lines must be very perceptible. There is no suggestion in the Butnaro patent of any steps that might be taken to reduce a user""s perception of the cue lines, to avoid the distraction that they would cause while carrying out any other visual task. For example, if a user is attempting to watch a movie while a passenger on an aircraft, the Butnaro system would likely interfere with the enjoyment of the movie by overlaying the very visible and distracting cue lines on the visual input, thereby detracting from the scenes portrayed on the movie screen or other display. Butnaro""s cue lines must be interpreted to determine the motion that has been sensed and require that a user be trained to interpret the motion and pay close attention. Accordingly, it appears that the user must directly xe2x80x9cseexe2x80x9d the cue lines to make the correct interpretation, or they will have little benefit. Intently watching a movie would interfere with the process of interpreting the cue lines if they are not so apparent, and the cue lines would thus provide little benefit.
No apparatus or technique is known that can provide an IVB truly corresponding to the sensations of a person""s vestibular system that does not interfere with other visual tasks, or which can be controlled by a user, or which can provide a benefit for a plurality of people. The present invention addresses these issues.
In accord with the present invention, a method is defined for preventing a person from experiencing motion sickness. In one embodiment of the method, a motion experienced by the person is sensed, producing a signal indicative of the motion. The signal is processed so as to produce a modified signal that is compensated to correspond to a perception of the motion by the vestibular system of the person. Using the modified signal, an IVB is produced that includes visual cues matched to the perception of the motion by the vestibular system. This IVB tends to compensate for any mismatch between the visual perceptions of the person and the perception of motion by the person""s vestibular system.
In one embodiment, the step of processing includes the step of employing a semicircular canal model to modify the signal to correspond to that of the vestibular system. In another embodiment, an otolith model is employed to modify the signal. In addition, the step of processing preferably includes the step of compensating the signal for the affects of individual differences in the perception of motion by the person""s vestibular system.
For certain applications of the present invention, the step of displaying includes the step of projecting the IVB onto a surface that at least partially surrounds the person. Furthermore, the step of displaying can include the step of displaying a grid of lines comprising the visual cues that match the perception of the motion by the vestibular system. The grid of lines includes a first set of lines and a second set of lines, with the first set of lines being generally orthogonal to the second set of lines. Alternatively, the step of displaying the IVB may include the step of displaying a plurality of lines comprising the visual cues. In this embodiment, the plurality of lines match the perception of the motion by the vestibular system.
It is understood that the person may be engaged in an activity that involves a visual task, such as watching a motion picture in which considerable motion occurs. In this case, the step of displaying the IVB is implemented so as to limit a distracting affect of the IVB on the person while engaging in the visual task. Thus, it is preferable that the person be enabled to control one or more variables that affect the perceptibility of the IVB. These variables include an absolute or relative luminance of the IVB on the display, a relative position, extent, or size of the IVB, and a position at which the IVB appears relative to a position at which the visual task appears. Another controllable variable is a timing with which the display of the IVB occurs. For example, the control can be employed to modify a periodic time interval during which the IVB is repetitively displayed, an interval between successive periodic displays of the IVB, and/or a rate at which the IVB is periodically displayed. Yet other variables controllable by a user include a relative position, an extent, and a size on a display at which the IVB is displayed, either a focus of the IVB relative to a display or a depth of focus of the IVB in the space visually perceived by the person, a color of the IVB, and a color contrast of the IVB relative to a background on which it is displayed.
If the visual task comprises the step of viewing an image in which substantial motion is portrayed, the method preferably further includes the step of displaying the IVB so as to avoid obscuring the image.
In other applications of the present invention, the step of displaying includes the step of moving the visual cues about in a manner consistent with the perception of the motion by the vestibular system of the person.
The step of sensing motion preferably comprises the step of sensing at least one of a linear position, an angular position, a linear velocity, an angular velocity, a linear acceleration, and an angular acceleration, to which the person is subject. (It is noted that in most applications, six degrees of freedom will be sensed, although in certain applications, sensing three degrees of freedom can provide the signals necessary to fully implement the IVB.) When processing the signal, the step of modifying the signal is preferably carried out as a function of a first time constant and a second time constant. In this embodiment, the second time constant is substantially longer in duration than the first time constant.
Another aspect of the present invention is directed to a system for preventing a person from experiencing motion sickness due to a difference between a visual sensory perception and a vestibular system perception of motion. The system preferably includes a memory in which a plurality of machine instructions are stored, a display, and at least one motion sensor that produces an output signal indicative of motion. A processor, which is coupled to the memory, the display, and the one or more motion sensors, executes the plurality of machine instructions. These machine instructions cause the processor to carryout functions that are generally consistent with the steps of the method discussed above.
Still other aspects of the present invention are directed to a method and system for reducing an adverse physiological reaction caused by differences in a visually perceived motion and a lack of motion as sensed by the internal vestibular system of a person. In these aspects of the invention, a fixed IVB comprising visual cues substantially corresponding to and consistent with the lack of motion sensed by the internal vestibular system of the person is provided and displayed relative to an image depicting substantial motion that is being viewed by the person. The display of the IVB occurs without obscuring the image. The visual cues, which are consistent with the lack of motion perceived by the internal vestibular system of the person, substantially reduce an adverse physiological reaction that would be experienced by the person if the IVB were not displayed.